# Copyright (C) 2020, 2021 Maxime Devos
# SPDX-License-Identifier: FSFAP
# Copying and distribution of this file, with or without modification,
# are permitted in any medium without royalty provided the copyright
# notice and this notice are preserved.  This file is offered as-is,
# without any warranty.

* TODO Version 0.1 [66%]
  The message queue code seems to work, there is a configuration
  parser, there is an API to the NSE client, so we almost have
  enough functionality for a demonstration ...
** DONE NSE client
   See examples/nse-web.scm
*** TODO A GUI (use guile-gnome)
    I made a web interface instead. (examples/web.scm)
*** DONE Load configuration from a file
** DONE Documentation [6/6]
*** DONE Concurrency basics
*** DONE Configuration section
    Incomplete but good enough.
*** DONE Network structures (incomplete)
*** DONE Communication with services
    Incomplete but decent?
*** DONE Estimation of size of network
*** DONE Installation instructions
    It became build instructions instead.
** TODO release
* (Guix) Substitutes over GNUnet

  Source files:
  + gnu/gnunet/scripts/download-store.scm
  + gnu/gnunet/scripts/publish-store.scm

  For previous attempts, see the guix mailing list
  and guile-gnunet.
** (Informative) How to use GNUnet FS without networking
   (At cost of not finding and sending content from / to
   other peers, of course)

   --no-grafts: its only for testing, so not spending any
   time with grafting is appreciated.

   #+BEGIN_SRC shell
     gnunetdir=`guix build gnunet --no-grafts`
     OLDPATH=$PATH
     PATH=$gnunetdir/bin:$gnunetdir/lib/gnunet/libexec:$PATH
     export PATH; hash
     gnunet-service-fs& gnunet-service-datastore&
     # Now have some fun without having to worry about
     # any potential security bugs!
   #+END_SRC

** DONE Publishing and fetching directories over GNUnet
   Example usage of publish-store:
   #+BEGIN_SRC shell
   guile -L . -e '(@ (gnu gnunet scripts publish-store) main)' -- -i `guix build hello`
   #+END_SRC

   Output:
   #+BEGIN_SRC fundamental
   Published at gnunet://fs/chk/DD0C9CM0EG1WPV5MQMWJ1B6QPVPR4R0DEZH8EW2BPASV9B9Q13P2TXWWX59RRBYGSS29TJEJE8WBNS97WQBDWHB7PGE0P21FMNYT83G.YVWEM5SWBHFQR1YFG8TAJD5J3284XDEK144BFY3B2F3BRD678T3HZPDTP7WB18A04N3HT4HC0SZZ6H71XQRVVZNZX2ZGJXXSJXD9KW0.24798 in gnunet-nix-archive-json/0 format
   #+END_SRC

   Example usage of download-store:
   #+BEGIN_SRC shell
     guile -L . -e '(@ (gnu gnunet scripts download-store) main)' -- -i $THE_GNUNET_URI -o t
     # Output: plenty of progress messages
     ls -R t
     # Success!
   #+END_SRC

** TODO Doing it *efficiently* [1/7]

   Some proposals (some forthe implementation, some of the directory format):

   * [ ] inline tiny files
   * [ ] parallel downloads
   * [ ] compress files
     Potential problem: if some peers use different compressors,
     then the files will be different --> more duplication on the network,
     less performant.
   * [ ] maybe use a binary format instead of JSON
   * [ ] factor out store names.

     Sometimes, changing a build dependency changes very little,
     perhaps only the embedded store names (e.g. when grafting). Maybe
     zero out the store references in the files, and mark the location
     anv value in the JSON container. This way, less data should be
     duplicated over the network (which should lead to better performance!).
   * [X] indexing (letting gnunet-service-fs refer to the store items by
     file name instead of inserting the store items in a database).
     This is done by default in the implementation for the
     gnunet-nix-archive-json/0 format. The former point ‘factor out store
     names’ will undo this, however (which could be resolved with
     a Hurd translator or FUSE filesystem).
   * [ ] contact 'gnunet-service-fs' directly, instead of calling a
     binary for each file.

** TODO Let Guix use a GNUnet substituter
   Make the substituters ‘pluggable’ somehow. Maybe allow
   to use the user's substituter, instead of the system's
   subsituter? (Practical for testing and experimentation.)

   Allow substitutes over http, ipfs, GNUnet, torrents (?).
   Insert some text about open coding here.

*** TODO support nar format [1/2]
    For easy use from Guix publish & substitute.

    - [X] for downloading to
    - [ ] for publishing from
      (probably not required, but nice for symmetry)

*** TODO make Guix publish extensible [2/3]

    - [X] write proposal
    - [X] submit proposal
    https://issues.guix.gnu.org/46214
      XXX not really happy about it ...
    - [ ] incorporate feedback for ‘narinfo generation hooks’,
          & proposal makes it upstream

*** TODO make Guix substitute extensible [3/4]

    - [X] write proposal
    - [X] make sure it actually works
      (test with publish-store and download-store)
      (currently rather crude)

      The HTTP substituter has been rewritten in terms
      of the proposal (and "make check" succeeds).  However,
      it hasn't been tested with a GNUnet or IPFS substituter.
    - [X] submit proposal
    https://issues.guix.gnu.org/46800
    - [ ] incorporate feedback & proposal makes it upstream

      Ludovic Courtes (12 Mar 2021):
      I don’t think my concerns are really addressed :-), but
      at the same time I think we need a playground for these things
      so we can actually grownew substitute methods like those you’ve
      been looking at.  Hmmm tricky!

*** TODO add gnunet extension to guix publish and guix substitute  [0/5]

    - [ ] clean up source code somewhat
    - [ ] release
    - [ ] submit package definitions
    - [ ] submit extra service definitions
    - [ ] incorporate feedback & code makes it upstream

** TODO (external) GNUnet service for Guix System

   Put it in a container to easy people's minds (including
   my own).
** TODO more TODO
   Maybe substitutes in GNS or something? Let's first
   address the former points.
